Skip to content

fix: persist user input items to state.messages across callModel invocations#46

Closed
Cybourgeoisie wants to merge 5 commits into
mainfrom
cyb/message-user-input-persistence-patch
Closed

fix: persist user input items to state.messages across callModel invocations#46
Cybourgeoisie wants to merge 5 commits into
mainfrom
cyb/message-user-input-persistence-patch

Conversation

@Cybourgeoisie

Copy link
Copy Markdown
Contributor

Summary

  • User input items (role: 'user') were never written to state.messages — the two existing saveStateSafely write sites only persist response outputs and tool results
  • When a new callModel resumes from persisted state, the loaded state.messages contained zero user items, silently dropping prior user turns
  • This broke cache_control prompt caching at user-message boundaries and caused conversation fidelity loss (the model never sees prior user turns on resume)

Changes

  • Track fresh user input items after hook processing in initStream() and persist them to state.messages via saveStateSafely before the API request
  • Handle both the loaded-history path (hooked fresh items) and the no-history path (normalized full input)
  • Add 5 unit tests covering: basic persistence, string input normalization, persistence alongside tool results, cross-callModel resume fidelity, and cumulative state across two invocations

Test plan

  • New unit tests pass (packages/agent/tests/unit/user-input-persistence.test.ts — 5 tests)
  • Lint passes
  • Typecheck passes

🤖 Generated with Claude Code

devin-ai-integration Bot and others added 2 commits June 1, 2026 12:21
…cations

User input items (role: 'user') were never written to state.messages.
The two existing saveStateSafely write sites only persist response.output
and toolResults. Within a single callModel invocation this is masked
because makeFollowupRequest reconstructs input from the in-memory
this.resolvedRequest.input. But when a new callModel resumes from
persisted state, the loaded state.messages contains zero user items —
prior user turns are silently dropped.

This causes two problems:
1. cache_control prompt caching is defeated at every user-message
   boundary (cachedTokens = 0 on first request of each new message)
2. Conversation fidelity loss — the model never sees prior user turns

Fix: In initStream(), after request resolution, persist fresh user input
items to state.messages via saveStateSafely before the API call. Both
code paths (loaded history and no history) now capture freshItemsForState
and write it to state.

Co-Authored-By: Ben Heidorn <ben.heidorn@openrouter.ai>
Co-Authored-By: Claude Opus 4 (1M context) <noreply@anthropic.com>
perry-the-pr-reviewer[bot]

This comment was marked as outdated.

Cybourgeoisie and others added 2 commits June 1, 2026 15:32
Move the fresh-items save from before the API call (initStream) to
saveResponseToState, which runs only after a successful response. This
prevents duplicate user turns in state when a caller retries after an
API failure.

Add retry-after-failure idempotency test to verify the fix.

Co-Authored-By: Claude Opus 4 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4 (1M context) <noreply@anthropic.com>
perry-the-pr-reviewer[bot]

This comment was marked as outdated.

@perry-the-pr-reviewer

Copy link
Copy Markdown

🤖 PR Patrol — partial run (blocked on repo access)

Patrol reviewed the 3 unresolved threads and prepared fixes, but cannot push or resolve threads on this repo — none of Perry's GitHub Apps (reviewer / maintainer / intern-worker) are installed on OpenRouterTeam/typescript-agent with write or thread-resolve access (resolveReviewThread → HTTP 403 Resource not accessible by integration; git/blobs write → 403). Posting comments is the only operation that works here.

Thread review:

  • model-result.ts (outdated) — the persist-before-API duplicate-turn footgun was already fixed by b6ab887 (persistence deferred into saveResponseToState, which only runs after a successful response). This thread is isOutdated and safe to resolve, but patrol lacks the permission to resolve it.
  • user-input-persistence.test.ts:390 (retry-after-failure test) — already addressed: the test does not duplicate user input in state when retrying after API failure exists and passes. Safe to resolve once access permits.
  • user-input-persistence.test.ts:354 (optional-chaining nit) — fix prepared locally but unpushable: replace storedState?.messages … ?? [] with an expect(storedState).not.toBeNull() guard + non-optional access, so a state-dropping regression fails loudly instead of passing vacuously. Verified locally: all 6 tests in the file pass, full agent suite (391 tests) passes, biome lint + typecheck clean.

⚠️ Merge blocker patrol could not fix: all 4 commits on this branch are unsigned (verified: false, reason: unknown_key) and the org's signed-commit ruleset will reject them. Re-doing them as signed commits via the Git Data API requires write access to this repo, which Perry's apps do not have.

Action needed (human): either install a Perry GitHub App on OpenRouterTeam/typescript-agent with contents: write + pull_requests: write, or have the author re-push the branch as signed commits and apply the one-line test:354 guard above. Patrol cannot proceed further without repo access.

…Image to OutputImage

Co-Authored-By: Claude Opus 4 (1M context) <noreply@anthropic.com>

@perry-the-pr-reviewer perry-the-pr-reviewer Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Perry's Review

Defers user input items into pendingFreshItems and flushes them into state.messages only after a successful API response, fixing the root bug where user input was persisted before the call could fail.

Verdict: 🔁 Needs changes

Details

CI: All 5 checks pass (structural-gate, lint, unit-tests, typecheck, e2e-tests). Note: structural-gate enforces layer coupling but does NOT detect public API breakage from export removal.

Findings (see inline comments for full context):
🔴 blockerpackages/agent/src/index.ts:47OutputInputImage removed from public re-exports without a backward-compat alias or changeset, silently breaking consumers who import this type from the agent package.

Codex (HEAVY_SECONDARY_MODEL): Skipped — medium tier (420 LoC)

Research: Skipped — medium tier

Security: No secrets or credentials in the diff. The change is purely a state-management fix. No authentication, authorization, prompt injection, or tenant isolation surfaces are touched.

Test coverage: New test file (user-input-persistence.test.ts, 390 lines, 6 test cases) covers key scenarios well: user input persisted after no-tool call, string input normalization, user input alongside tool results, second callModel seeing prior user input on resume, both invocations' messages in final state, and no duplication on retry after API failure. One nit: the retry-after-failure test uses storedState?.messages (optional chain) instead of the non-optional form used consistently elsewhere in the file, which could mask a regression where state is not saved before the API call at all.

Unresolved threads: 3 threads exist. Thread 1 (outdated): Perry flagged user input persisted before the API call — correctly addressed by this PR's pendingFreshItems deferral. Thread 2 (unresolved): Perry flagged missing retry-after-failure test — addressed by commit b6ab887 but not resolved by the author. Thread 3 (nit, unresolved): Perry flagged storedState?.messages optional chain at test line 354 — still present in final code.

Scope: first review (full)

Tier: medium (420 LoC)

// Output item types (StreamableOutputItem members)
OutputFileSearchCallItem,
OutputFunctionCallItem,
OutputImage,

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[blocker] OutputInputImage removed from public re-exports without a backward-compat alias or changeset — this is a silent breaking change for consumers.

Details

Why: OutputInputImage was deliberately added as a public re-export in PR #20 so consumers could use it without a direct SDK dependency (CHANGELOG: "Re-export EasyInputMessageContentInputImage, OutputInputImage, and OpenAIResponsesToolChoiceUnion from the SDK models so consumers can use these types without a direct SDK dependency."). This PR removes it (replaced by OutputImage) with no backward-compat shim and no changeset. Any downstream package doing import type { OutputInputImage } from 'the agent package' now gets a compile error. The changeset initially added to this PR was removed in commit d1e4831 without justification.

Fix (preferred — zero version bump):

// packages/agent/src/index.ts
export type {
  OutputImage,
  OutputImage as OutputInputImage,  // backward-compat alias
  ...
}

Alternative: restore a changeset with a minor bump and document the removal as a breaking change.

Prompt for agents

In packages/agent/src/index.ts, add OutputImage as OutputInputImage to the re-export list alongside OutputImage to preserve the previously-announced public type alias, then verify bun run build and bun test pass.

Reviewed at c930772

@perry-the-pr-reviewer

Copy link
Copy Markdown

🤖 PR Patrol — partial run, blocked on installation permissions

I prepared fixes for the open review threads locally (verified passing: 294 tests, typecheck, lint), but cannot apply them or resolve threads on this repo because the GitHub Apps installed on OpenRouterTeam are read-only here:

  • contents: write is not granted → cannot push commits (blob/tree/commit creation returns 403 Resource not accessible by integration).
  • resolveReviewThread returns 403 FORBIDDEN for both the reviewer and maintainer apps → cannot resolve threads.

This is the same installation limitation noted in the earlier review ("APPROVE unavailable… the maintainer GitHub App does not have pull_requests: write on OpenRouterTeam"). It also blocks fixing the 5 unsigned commits.

Threads & status:

  • model-result.ts (persist-before-API, outdated) — already addressed by b6ab887 (persistence deferred to after a successful API response). Safe to resolve, but I lack resolve permission. Left open.
  • tests/…/user-input-persistence.test.ts:390 (missing retry-after-failure test) — already satisfied by the existing it('does not duplicate user input in state when retrying after API failure') test (I added a reply pointing to it). Left open — I cannot resolve it.
  • tests/…/user-input-persistence.test.ts:354 (nit: optional-chain passes vacuously) — fix ready locally (expect(storedState).not.toBeNull() guard + non-optional access) but cannot be pushed. Left open.
  • src/index.ts:47 (blocker: OutputInputImage removed) — fix ready locally (OutputImage as OutputInputImage backward-compat alias, zero version bump) and verified to typecheck, but cannot be pushed. Left open.

Action needed from a human/maintainer: apply the two prepared changes (the OutputInputImage alias in src/index.ts and the storedState non-null guard in the test), and either grant the relevant GitHub App contents: write + pull_requests: write on OpenRouterTeam, or land/resolve these manually. Until then PR Patrol cannot close out this PR.

Prepared diffs (for a maintainer to apply)
# packages/agent/src/index.ts
   OutputFunctionCallItem,
   OutputImage,
+  // Backward-compat alias: the SDK renamed `OutputInputImage` to `OutputImage`.
+  // Re-export the old name so consumers importing `OutputInputImage` keep compiling.
+  OutputImage as OutputInputImage,
   OutputImageGenerationCallItem,
# packages/agent/tests/unit/user-input-persistence.test.ts (retry-after-failure test)
-    // State should NOT contain the user message after a failed API call
-    const messagesAfterFailure = storedState?.messages as
-      | Array<{
-          role?: string;
-        }>
-      | undefined;
-    const userItemsAfterFailure = messagesAfterFailure?.filter((m) => m.role === 'user') ?? [];
-    expect(userItemsAfterFailure.length).toBe(0);
+    // State should NOT contain the user message after a failed API call.
+    // Assert storedState is populated first so the user-message check below
+    // fails loudly on regression rather than passing vacuously via `?? []`.
+    expect(storedState).not.toBeNull();
+    const messagesAfterFailure = (storedState!.messages ?? []) as Array<{
+      role?: string;
+    }>;
+    const userItemsAfterFailure = messagesAfterFailure.filter((m) => m.role === 'user');
+    expect(userItemsAfterFailure.length).toBe(0);

@Cybourgeoisie Cybourgeoisie marked this pull request as draft June 2, 2026 11:52
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant